Coronavirus US - USA Facts
davegoblue
2022-09-13
Background
This module builds on code contained in Coronavirus_Statistics_USAF_v006.Rmd. This file includes the latest code for analyzing data from USA Facts. USA Facts maintains data on cases and deaths by county for coronavirus in the US. Downloaded data are unique by county with date as a column and a separate file for each of cases, deaths, and population.
The intent of this code is to source updated functions that allow for readRunUSAFacts() to be run to obtain, read, process, and analyze data from USA Facts.
Sourcing Functions
The tidyverse library is loaded, and the functions used for CDC daily processing are sourced. Additionally, specific functions for USA Facts are also sourced:
library(tidyverse)## ── Attaching packages ─────────────────────────────────────── tidyverse 1.3.2 ──
## ✔ ggplot2 3.3.6 ✔ purrr 0.3.4
## ✔ tibble 3.1.8 ✔ dplyr 1.0.9
## ✔ tidyr 1.2.0 ✔ stringr 1.4.0
## ✔ readr 2.1.2 ✔ forcats 0.5.1
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag() masks stats::lag()# Functions are available in source file
source("./Generic_Added_Utility_Functions_202105_v001.R")
source("./Coronavirus_CDC_Daily_Functions_v002.R")
source("./Coronavirus_USAF_Functions_v001.R")Further, the mapping file specific to USA Facts is sourced:
source("./Coronavirus_USAF_Default_Mappings_v002.R")Updated functions for diagnoseClusters(), createDetailedSummaries(), createSummary(), and helperSummaryMap() are included in Coronavirus_USAF_Functions_v001.R. These functions should be checked for consistency with state-level data with just a single copy kept later.
Example Process
The latest county-level burden data are downloaded:
readList <- list("usafCase"="./RInputFiles/Coronavirus/covid_confirmed_usafacts_downloaded_20220913.csv",
"usafDeath"="./RInputFiles/Coronavirus/covid_deaths_usafacts_downloaded_20220913.csv"
)
compareList <- list("usafCase"=readFromRDS("cty_newdata_20220807")$dfRaw$usafCase,
"usafDeath"=readFromRDS("cty_newdata_20220807")$dfRaw$usafDeath
)
# Use existing clusters
cty_newdata_20220913 <- readRunUSAFacts(maxDate="2022-09-11",
downloadTo=lapply(readList,
FUN=function(x) if(file.exists(x)) NA else x
),
readFrom=readList,
compareFile=compareList,
writeLog="./RInputFiles/Coronavirus/USAF_NewData_20220913_chk_v005.log",
ovrwriteLog=TRUE,
useClusters=readFromRDS("cty_newdata_20210813")$useClusters,
skipAssessmentPlots=FALSE,
brewPalette="Paired"
)##
## No file has been downloaded, will use existing file: ./RInputFiles/Coronavirus/covid_confirmed_usafacts_downloaded_20220913.csv## Rows: 3193 Columns: 962
## ── Column specification ────────────────────────────────────────────────────────
## Delimiter: ","
## chr (3): County Name, State, StateFIPS
## dbl (959): countyFIPS, 2020-01-22, 2020-01-23, 2020-01-24, 2020-01-25, 2020-...
##
## ℹ Use `spec()` to retrieve the full column specification for this data.
## ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.##
## *** File has been checked for uniqueness by: countyFIPS countyName state stateFIPS
##
##
## *** File has been checked for uniqueness by: countyFIPS stateFIPS date
##
##
## Checking for similarity of: column names
## In reference but not in current:
## In current but not in reference:
##
## Checking for similarity of: date
## In reference but not in current: 0
## In current but not in reference: 33
## Detailed differences available in: ./RInputFiles/Coronavirus/USAF_NewData_20220913_chk_v005.log
##
## Checking for similarity of: county
## In reference but not in current:
## In current but not in reference:
##
##
## ***Differences of at least 5 and at least 5%
##
## 2 records
## Detailed output available in log: ./RInputFiles/Coronavirus/USAF_NewData_20220913_chk_v005.log##
##
## ***Differences of at least 0 and at least 0.1%
##
## 128 records
## Detailed output available in log: ./RInputFiles/Coronavirus/USAF_NewData_20220913_chk_v005.log## Rows: 3193 Columns: 962
## ── Column specification ────────────────────────────────────────────────────────
## Delimiter: ","
## chr (3): County Name, State, StateFIPS
## dbl (959): countyFIPS, 2020-01-22, 2020-01-23, 2020-01-24, 2020-01-25, 2020-...
##
## ℹ Use `spec()` to retrieve the full column specification for this data.
## ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.
##
## *** File has been checked for uniqueness by: countyFIPS countyName state stateFIPS
##
##
## *** File has been checked for uniqueness by: countyFIPS stateFIPS date
##
##
## Checking for similarity of: column names
## In reference but not in current:
## In current but not in reference:
##
## Checking for similarity of: date
## In reference but not in current: 0
## In current but not in reference: 33
## Detailed differences available in: ./RInputFiles/Coronavirus/USAF_NewData_20220913_chk_v005.log
##
## Checking for similarity of: county
## In reference but not in current:
## In current but not in reference:
##
##
## ***Differences of at least 5 and at least 5%
##
## 3 records
## Detailed output available in log: ./RInputFiles/Coronavirus/USAF_NewData_20220913_chk_v005.log
##
##
## ***Differences of at least 0 and at least 0.1%
##
## 108 records
## Detailed output available in log: ./RInputFiles/Coronavirus/USAF_NewData_20220913_chk_v005.log
##
##
## Column sums before and after applying filtering rules:
## # A tibble: 3 × 4
## isType cases new_cases n
## <chr> <dbl> <dbl> <dbl>
## 1 before 3.45e+10 93305986 3058894
## 2 after 3.43e+10 91158859 3010036
## 3 pctchg 7.16e- 3 0.0230 0.0160
##
##
## Column sums before and after applying filtering rules:
## # A tibble: 3 × 4
## isType deaths new_deaths n
## <chr> <dbl> <dbl> <dbl>
## 1 before 5.12e+8 1034939 3058894
## 2 after 4.95e+8 964043 3010036
## 3 pctchg 3.23e-2 0.0685 0.0160
## NULL
# Plot all counties based on closest cluster
sparseCountyClusterMap(cty_newdata_20220913$useClusters,
caption="Includes only counties with 25k+ population",
brewPalette="viridis"
)
# Save the refreshed file
saveToRDS(cty_newdata_20220913, ovrWriteError=FALSE)Vaccines data are also updated, though the process needs to integrate previous data:
cty_vaxdata_20220914 <- processCountyVaccines(loc="./RInputFiles/Coronavirus/county_vaccine_20220914.csv",
ctyList=cty_newdata_20220913,
minDateCD=c("2022-06-09", "2022-06-09"),
maxDateCD="2022-09-01"
)## Rows: 78961 Columns: 72
## ── Column specification ────────────────────────────────────────────────────────
## Delimiter: ","
## chr (6): Date, FIPS, Recip_County, Recip_State, SVI_CTGY, Metro_status
## dbl (66): MMWR_week, Completeness_pct, Administered_Dose1_Recip, Administere...
##
## ℹ Use `spec()` to retrieve the full column specification for this data.
## ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.##
## Records from other than 50 states and DC:
## # A tibble: 9 × 2
## state n
## <chr> <int>
## 1 AS 24
## 2 FM 25
## 3 GU 48
## 4 MH 24
## 5 MP 24
## 6 PR 1901
## 7 PW 24
## 8 VI 96
## 9 <NA> 17
## Warning: Removed 16 rows containing non-finite values (stat_boxplot).
## Warning: Removed 16 rows containing non-finite values (stat_boxplot).
## Warning: Removed 16 rows containing non-finite values (stat_boxplot).
##
## Count of NA records by column
## state FIPS popgte65_minpop popgte65_maxpop popgte65_nnA
## 0 0 0 0 0
## n
## 0
##
## Records where minimum and maximum population differ# A tibble: 0 × 5
## # … with 5 variables: state <chr>, FIPS <chr>, age <chr>, minpop <dbl>,
## # maxpop <dbl>
## # ℹ Use `colnames()` to see all variable names
##
##
##
## Will run with parameters:
## burdenVar: cpm dpm
## vaxVar: vxcpoppct vxcpoppct
## minDateCD: 2022-06-09 2022-06-09
## maxDateCD: 2022-09-01 2022-09-01## `geom_smooth()` using formula 'y ~ x'## Warning: Removed 16 rows containing non-finite values (stat_smooth).## Warning: Removed 16 rows containing missing values (geom_point).
## `geom_smooth()` using formula 'y ~ x'## Warning: Removed 16 rows containing non-finite values (stat_smooth).
## Removed 16 rows containing missing values (geom_point).
##
## Call:
## lm(formula = get(burdenVar) ~ vaxMetric, data = dfReg, weights = pop)
##
## Weighted Residuals:
## Min 1Q Median 3Q Max
## -289154037 -1505413 -90865 1548407 62031339
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 20807.226 2187.230 9.513 <2e-16 ***
## vaxMetric 8.061 33.809 0.238 0.812
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 7367000 on 3124 degrees of freedom
## (16 observations deleted due to missingness)
## Multiple R-squared: 1.82e-05, Adjusted R-squared: -0.0003019
## F-statistic: 0.05684 on 1 and 3124 DF, p-value: 0.8116
##
##
## Call:
## lm(formula = get(burdenVar) ~ vaxMetric * type + 0 - vaxMetric,
## data = dfReg, weights = pop)
##
## Weighted Residuals:
## Min 1Q Median 3Q Max
## -288389946 -1463009 -64586 1597997 65055859
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## type<25k 12176.47 7942.37 1.533 0.125352
## type>500k 29162.62 4682.01 6.229 5.34e-10 ***
## type100k-500k 17769.84 4666.92 3.808 0.000143 ***
## type25k-100k 17521.42 5263.39 3.329 0.000882 ***
## vaxMetric:type<25k 175.75 159.92 1.099 0.271861
## vaxMetric:type>500k -110.85 66.36 -1.670 0.094933 .
## vaxMetric:type100k-500k 60.38 74.96 0.805 0.420630
## vaxMetric:type25k-100k 66.14 99.33 0.666 0.505574
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 7368000 on 3118 degrees of freedom
## (16 observations deleted due to missingness)
## Multiple R-squared: 0.4677, Adjusted R-squared: 0.4663
## F-statistic: 342.4 on 8 and 3118 DF, p-value: < 2.2e-16## `geom_smooth()` using formula 'y ~ x'## Warning: Removed 16 rows containing non-finite values (stat_smooth).
## Removed 16 rows containing missing values (geom_point).
## `geom_smooth()` using formula 'y ~ x'## Warning: Removed 16 rows containing non-finite values (stat_smooth).
## Removed 16 rows containing missing values (geom_point).
##
## Call:
## lm(formula = get(burdenVar) ~ vaxMetric, data = dfReg, weights = pop)
##
## Weighted Residuals:
## Min 1Q Median 3Q Max
## -3529509 -12224 -2495 14425 697276
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 298.9805 29.5946 10.103 <2e-16 ***
## vaxMetric -4.0106 0.4575 -8.767 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 99680 on 3124 degrees of freedom
## (16 observations deleted due to missingness)
## Multiple R-squared: 0.02401, Adjusted R-squared: 0.0237
## F-statistic: 76.86 on 1 and 3124 DF, p-value: < 2.2e-16
##
##
## Call:
## lm(formula = get(burdenVar) ~ vaxMetric * type + 0 - vaxMetric,
## data = dfReg, weights = pop)
##
## Weighted Residuals:
## Min 1Q Median 3Q Max
## -3480323 -13174 -5106 10332 707402
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## type<25k 130.2314 107.0422 1.217 0.22384
## type>500k 383.7067 63.1011 6.081 1.34e-09 ***
## type100k-500k 99.6779 62.8978 1.585 0.11312
## type25k-100k 207.0272 70.9366 2.918 0.00354 **
## vaxMetric:type<25k -0.3702 2.1553 -0.172 0.86365
## vaxMetric:type>500k -5.4661 0.8943 -6.112 1.11e-09 ***
## vaxMetric:type100k-500k -0.5040 1.0103 -0.499 0.61791
## vaxMetric:type25k-100k -1.8241 1.3388 -1.362 0.17314
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 99300 on 3118 degrees of freedom
## (16 observations deleted due to missingness)
## Multiple R-squared: 0.05224, Adjusted R-squared: 0.04981
## F-statistic: 21.48 on 8 and 3118 DF, p-value: < 2.2e-16# Save the refreshed file
saveToRDS(cty_vaxdata_20220914, ovrWriteError=FALSE)County-level data are post-processed:
cty_postdata_20220913 <- postProcessCountyData(lstCtyBurden=cty_newdata_20220913$dfPerCapita,
lstCtyVax=cty_vaxdata_20220914$vaxFix,
lstState=readFromRDS("cdc_daily_220902")$dfPerCapita
)##
## Parameter maxDate is: 2022-09-01
# Save the refreshed file
saveToRDS(cty_postdata_20220913, ovrWriteError=FALSE)Additional post-processing steps are run:
# Step 1a: Burden comparisons for aggregated states
additionalCountyPostProcess(cty_postdata_20220913, p1CompareStates=c(state.abb, "DC"), p1AggData=TRUE)## Warning: Removed 6 row(s) containing missing values (geom_path).
# Step 1: Burden aggregation for key states
# Step 2: vaccine comparisons
# Step 3: Scoring updates (and errors)
# Step 4: New rolling data (28-day default with ceilings 50000 CPM, 500 DPM)
additionalCountyPostProcess(cty_postdata_20220913,
p1CompareStates=c("GA", "FL", "NE"),
p2VaxStates=c("MA", "HI", "TX", "VA", "VT", "GA", "CO", "SD"),
p3VaxTimes=sort(c("2022-01-01", "2022-08-31")),
p4DF=cty_newdata_20220913$dfPerCapita
)## Warning: Removed 6 row(s) containing missing values (geom_path).
## Warning: Removed 6 row(s) containing missing values (geom_path).
## Warning: Removed 6 row(s) containing missing values (geom_path).
## Warning: Removed 1 row(s) containing missing values (geom_path).
Additional plots are also updated:
# Creating the vaccine and burden data
tmpVaxBurden <- createVaxBurdenData(lstVax=cty_vaxdata_20220914, lstBurden=cty_newdata_20220913)
# Nationwide aggregation, excluding problem states
problemStates <- c("VA", "TX", "SD", "HI", "GA", "CO", "GA", "FL", "NE", "VA")
useStates <- state.abb
tmpCountyList <- tmpVaxBurden$ctyPop %>%
filter(state %in% useStates, !(state %in% problemStates)) %>%
left_join(filter(tmpVaxBurden$dfVaxBurden, name=="vxcpoppct", date==max(date)), by="countyFIPS") %>%
mutate(vaxPct=percent_rank(value),
vaxBucket=case_when(vaxPct <= .25 ~ "3. Low", vaxPct >= .75 ~ "1. High", TRUE ~ "2. Medium")
) %>%
split(f=.$vaxBucket)
# Plot of absolute burden
plotVaxBurdenData(tmpVaxBurden,
ctyPlot=lapply(tmpCountyList, FUN=function(x) x %>% rename(bucket=vaxBucket)),
plotTitle="Counties in states with continuous county data"
)
# Plots of burden relative to May 2021
plotVaxBurdenData(tmpVaxBurden,
ctyPlot=lapply(tmpCountyList, FUN=function(x) x %>% rename(bucket=vaxBucket)),
plotTitle="Counties in states with continuous county data",
scaleToDate="2021-05-01"
)
Multiple data anomalies need to be addressed - vaccines data covering only a short time period, counties significantly revising burden downwards, etc.